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Evaluating Narrative Simulation as Instructional Design for Potential to Impact 

Bias and Discrimination 

Nancye McCrary 
Joan M. Mazur 
University of Kentucky 



Abstract 

Simulation exercise as instruction has a long history in skill development involving performance tasks. 
Traditionally physical simulators have been used in diverse fields such as military flight instruction and 
industrial training. More recently paper pencil simulation exercises have employed narrative in 
instruction aimed at solutions to work place problems^ such as sexual harassment. This evolution has led to 
a refinement of principles of simulation exercise design, as well as evidence for specific problem 
characteristics that indicate potential effectiveness in use of simulations. Issues such as racism, anti- 
Semitism, sexism, disability, and homophobia present instructional challenges due to the covert and rigid 
nature of some attitudes and beliefs. This paper reports on the development, design, and initial field testing 
of a narrative simulation focused on homophobia and adolescent suicide. Desired learning outcomes that 
focus on attitudes, beliefs, and biases may lend themselves to further evolution of narrative simulation in 
creating contexts in which to examine deeply held beliefs. 

Narrative Simulation as an Instructional Design Strategy 

Simulation exercise as instruction has a long history in skill development involving performance tasks. 
Traditionally physical simulators have been used in military flight instruction, industrial training, and an array of 
other performance-based instructional problems. More recently paper & pencil simulation exercises have employed 
nairative in instruction aimed at accident prevention, farm safety, and solutions to work place problems, such as 
sexual harassment. This evolution has led to a refinement of principles of simulation exercise design, as well as 
evidence for specific problem characteristics, which indicate potential effectiveness in use of simulations. Issues such 
as racism, sexism, anti-Semitism, disability, and homophobia present instructional challenges due to the covert and 
rigid nature of some attitudes and beliefs. Group discussion is one powerful way of beginning to examine beliefs and 
synthesize new information regarding biases. Narrative simulation has the potential to initiate such discourse by 
providing less threatening contexts in which to think about and express deeply held beliefs. As a result it provides 
exposure to alternative ways of thinking. Desired learning outcomes that relate to attitudes, beliefs, and biases may 
well lend themselves to further evolution of narrative simulation as an instructional design strategy. 

Based on the work of Bruner (1986), Cole (1994), Howard (1991), Sarbin (1986), and others, this 
simulation exercise was developed as instructional material for presenting content on homophobia related to 
adolescent sexual orientation. '"This just isf" Jeff s Story uses a narrative approach to engender discourse and to 
inform regarding this difficult and prohibitive topic. It specifically deals with adolescent sexual orientation, 
homophobia, and potential suicide. It is a powerful conten^rary culture tale that offers opportunity for situated 
presentation of information (Lave and Wenger, 1991, Schell and Black, 1997, Young, 1993). The purpose of this 
paper is to describe the design and preliminary evaluation of this innovative narrative simulation. The evaluation 
utilizes data obtained through expert feedback, exercise field tests, individual and focus group interviews, and 
observation as basis for informing continued development of this design. 

"This Just is" Jeffs Story is a narrative simulation designed for target groups of parents, helping 
professionals, and adolescents. The story centers on a fourteen-year-old boy discovering his own honx)sexuality and 
struggling with problems of discrimination, violence, and suicidal thinking. This true story was adapted from a web 
page, developed by a parent who was faced with these issues. The title "This Just is" was adopted from a statement 
found in her son's journal after his suicide. 2 

The grand scheme was to explore instructional possibilities relative to constructivist theory. Narrative 
simulation provides opportunity to teach through problem-solving practice in real world environments (Duffy and 
Cunningham in Jonassen Ed., 1996). The overall intent was concerned with exploring instruction designed to impact 
socially constructed attitudes, creative problem solving, and independent thinking skills. Simulation exercise with its* 
imperatives for “real world” learning, veracity, and practice before the fact, as opposed to retrospective methods 
such as case study (Cole, 1997), seemed an appropriate place to begin. The narrative simulation approach 
increasingly involves the user in the story by virtue of question & answer decision points embedded throughout. 



2 http://members.tripod.com/-clavtolv/Biirs Story 
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We began this project with a broad question regarding whether instruction can be effectively designed to 
decrease bias and discrimination. Questions regarding bias, stereotyping, and attitudinal change appear to be some of 
the most challenging issues faced by contenporary educators (Boyer, 1995, Viu, 1990). An initial single focus on 
homophobia was chosen because discrimination towards gays is so pervasive in contenporary American culture 
(Helminiak, 1994). Open expression of homophobia has been painfully demonstrated in the most recent well- 
publicized case of Matthew Shepard, who was brutally beaten and hung on a fence to die, solely because of his 
homosexuality (Barrett, 1998). Furthermore, discrimination towards homosexuals, in most cases, is still legally 
condoned in this country (Plaster, 1998, Stachelberg, 1998). Relative to public education, Sandra Prettyman writes, 
"Creating a forum for discussion and an awareness of the issue [homophobia] will help promote student safely in 
schools and will help students begin to recognize and critique the oppression that many different groups of people 
face in our country today (1997, p. 93)." The long-range goal for this project is to develop a series of narrative 
simulation exercises that will impact various kinds of discrimination, both in pencil/paper and multimedia formats. 

Promoting Tolerance and Inclusion through Instruction 

While tolerance and inclusion are often recognized as priority issues in various instructional contexts 
(schools, business and industry, and the military), problems regarding how to promote sensitivity to difference with 
regard to race, culture, religion, gender, disability, and sexual orientation persist. Most trainers and educators realize 
that addressing these issues in general, rather than p>ersonal terms, has proved to be a superficial and ineffective 
approach to problems that have serious legal and moral consequences. It has been particularly difficult to address 
these issues in engaging and meaningful instructional ways. This has been partly due to the fact that instructional 
materials and approaches, designed to engender personal reflection and discourse on these difficult topics, are 
limited, often nonexistent (Prettyman, 1997). 

However, there is an emerging body of qualitative literature that seeks to address these issues. The 1998 
journal of Qualitative Studies in Education 11 (2) provides an editor's introduction that signals "strong support of 
lesbian and gay scholars and scholarship." The editors say "...we feel that all educators need to make known publicly 
their support of lesbians and gays in order to aid this group in their struggles (which really should be our struggle, 
too) to gain basic human rights." William Tierney, author of the book. Academic outlaws: Queer theory and cultural 
studies in the academy , says, in the process of studying a faculty member who died of AIDS, he began to understand 
"...the intense dilemmas that individuals face in coming to accept those parts of themselves that society most 
vociferously aims to deny.. .1 struggle to develop pedagogies that enable students to develop their own voice rather 
than speak with one voice (Tierney, 1998, p. 133)." Robert Rhoads recently conpleted a "two-year ethnographic 
study designed to better understand the coming-out experiences of gay and bisexual college men (Rhoads, 1997, 
p.8)." Chasnoff and Cohen have published an instructional film. It's elementary: Talking about gayjssues in schod 
(NewDay Films, 1997), which is reviewed by Prettyman in Educational Studies (1997, pp. 92-98). Catherine 
Raissiguier (1997) offers a study of 13 self-identified homosexual university students. Raissiguier (1997) notes that 
while earlier studies explored the "anomaly" of homosexual attraction and behavior it is relatively recent that 
homosexual youth have been studied as members of a collective group (pp.34-35). These studies, related articles, 
reviews, and editor's notes provide a base for the development of instructional materials and media designed to 
promote a deeper, and hopefully, richer understanding of all types of difference, particularly that of sexual 
orientation. 

Affecting Beliefs and Behavior through Narrative 

Walter Fisher (1995) offers well-articulated foundation for use of narrative to integrate the didactic and 
non-didactic (affective) aspects of instruction. “Narration (as paradigm) respects reason... as including metaphor as 
well as argument, as inextricably bound to values, and as historical and contextual (p.l73)." Fisher reminds us that 
objectivist thinking is rooted in concepts, methods, and hypotheses, which are historically grounded in myths. He 
adds, “objectivist thinking and discourse cannot advance without recourse to imagination (p.l74)". The 
inplications, for purposes of this project are to design instruction, which fills the space beyond— the gapping hole— as 
it were, left by the singular focus of classical science. That is, instruction designed with a primary focus on values in 
the context of contemporary culture. Instructional narrative that combines didactic information and narrative may be 
especially powerful with regard to the affective domain (Vitz, 1990). 

Henry Cole (1997) has organized and articulated issues regarding instructional narrative simulation through 
his efforts in farming and mining safety. While emphasizing seemingly traditional performance outcomes. Cole has 
also provided important clarity on the issue of inpacting learner attitudes and beliefs. Cole constructs solid 
theoretical foundation for use of narrative simulation in education. He states: “The robustness of the life stories 
people live, observe, and communicate about cannot be captured in a meaningful way with only the abstract 
principles of paradigmatic thinking (p.332)." Qne important contribution Cole makes is a narrative conceptual 
model, which he describes as “a cultural, cognitive, and behavioral model... of beliefs and... behavior (p.333)". In 
this model culture (stories heard, stories lived, and stories told) contributes to cognition (attitudes, beliefs, meaning. 



and knowledge), which in-tum contributes to conduct (behaviors, actions) that results in consequences (effects, 
outcomes). While the process is linear, insights gained through it often fold back into an interplay of the whole. 

Cole's model illustrates the relationship between narrative, cognition, behavior, and consequence. One point that he 
stresses is that narrative simulation can be an effective means to vicariously experience consequences and may even 
have power to change behavior before the fact. In Cole's original model consequences necessarily follow from 
narrative, cognition, and behavior. This model is pertinent to this development project with one exception. That is, 
we would add some indication that only when the social is made personal do consequences necessarily result from 
this process (see figure 1). This is an effort to move Cole's model a step further for specific consideration of issues of 
social values such as discriminatory behavior. This is to say that unless the cultural collective consequences are made 
personal through experience (vicarious or real); they may have no impact, behaviorally speaking, on individual 
attitudes and beliefs underlying discrimination. "Bruner (1986, 1990a) notes that a universal problem in education is 
the matter of translating socially relevant infonnation into information that is personally relevant for the individual 
(Cole, 1997, p.334)." Narrative form has the power to make the socially relevant nx)re personal for the learner 
(Bruner, 1986, Cole, 1997, Vitz, 1990). 



Figure 1 . Cultural, Cognitive, and Behavioral Model of Bias and Discrimination 




"The kind of meaning the narrative conveys about human existence requires the use of discourse, which can 
be differentiated from mere collection of words or sentences. A discourse is an integration of sentences that produces 
a global meaning that is more than that contained in the sentences... (Polkinghome, 1988, p. 31)." In addition to 
personalizing dilemmas and decisions, narrative also has important potential to stimulate group discussion. Bruner 
(1996) writes “...we form a representation of the world as much from what we learn about it through others as from 
responding to events in the world directly (p.l65)." This concept provides an important frame for instruction ainied 
at impacting the affective domain. We often develop attitudes, beliefs, and biases based on what we learn from others 
(individually and collectively) rather than actual experienced events. This is especially true with regard to 
homophobia and racism. Usually personal experience with those discriminated against is limited. Moreover, even 
when we have opportunities for such experiences, these experiences are filtered through pre-existing cultural and 
personal belief systems. Isolated experiences serve to either confirm pre-existing notions or stand as rare exceptions. 
Rarely do these exceptions impact deeply held beliefs. However, since experiences that result in bias are developed 
and conveyed largely through the vehicle of narrative, it is quite appropriate to look to forms of instructional 
narrative to change such beliefs. Sarbin (1986) contends that as human beings we make moral decisions according to 
narrative structures. As such, the use of stories has an enduring, if not prevailing, history as instructional practice. 

Combining Narrative Structure and Simulation 

Some important findings have emerged from research with narrative simulation exercises. Stories, when 
used in this manner, provide opportunity for critical thinking and decision making. As narrative simulation stories 
unfold, dilemmas develop which engage the learner in real world problems and relative solutions. The narrative 
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exercise requires gathering information, interpreting data, making choices, remembering and utilizing relevant 
information, and actually interacting with the plot of the story (Cole 1997, Howard 1991, Omer 1996, Schell and 
Black 1997, Vitz 1990, Young 1993). The learner literally becomes a character in an unfolding tale. As such, he or 
she is more likely to discuss beliefs, attitudes, and biases, otherwise considered taboo. Most would agree that bias 
and discrimination are serious problems, which tend to arrest individual and cultural growth and development. 
However, the tendency of human nature is to view these issues as other people's problems (Bennett, 1998). As an 
instructional strategy, situated cognition has been used to relate subject matter to the concerns of learners (Shor, 
1987). Situated learning involves immersing learners in activities and environments that facilitate ability to make 
meaning (Lave and Wenger, 1991). Situated learning typically involves four guiding principles: (1) learning is 
grounded in everyday activities; (2) knowledge is acquired situationally and transfers to similar situations; (3) 
learning is the result of a social process involving certain ways of thinking, perceiving, problem solving, and 
interacting; and (4) learning exists in complex social environments made up of actors, actions, and situations 
(Anderson, Reder, and Simon 1996). Dilemmas rather than content drive this type of learning. Certain problems are 
presented that serve to challenge the simulation user. 

"77z/5 just is*' Jeffs Story combines the aesthetic and affective power of narrative with the situated cognition 
of simulation by re-creating a real-life situation and utilizing strategically placed decision points. It involves the user 
as a character in an unfolding story, providing opportunity to think through problems, respond to questions, and 
resolve dilemmas as the story progresses. Additional information in the form of statistics, news media excerpts, 
history, and commentary is provided throughout the narrative. These appear at the bottom of each page and are 
distinguished from the narrative by use of text boxes. Users may choose to read any or all of this additional 
information. 

Development of the Narrative Simulation: '*This just is/" Jeffs Story 
The Search for a Story 

The imperatives of veracity and fidelity for simulation effectiveness led to an initial search for a true story. 
While it is possible to create a narrative with truth-likeness, it seemed that it would be more powerful to use an actual 
contemporary story. The power of contemporary, true story narrative to support reflection and change has been 
recently documented (Bliss and Mazur, 1996). Additionally, true story cases combined with facilitated discussion of 
difficult topics can create conditions for conversation and community (Mazur and Bliss, 1995). While Bruner 
(1990) and Grossman (1992) suggest that verisimilitude or plausibility is sufficient in narrative, Phillips (1994) 
argues for the use of true stories “when something important hangs in the balance (p. 17)." Moreover, evidence 
exists that the true story case provides for intense engagennent and personal involvement in the story (Bliss and 
Mazur, 1995). 

Bill's Story appeared in the form of a web page that was dedicated to prevention of teenage suicide and 
violence towards gays. Bill Clayton's mother developed this web page, after his death. She was contacted and 
generously agreed to assist with the content and design of this instructional package. Thus, Gabi Clayton is an 
important collaboratpr on this project, offering editorial revisions and support along the way. For example, she 
suggested that the use of the term lifestyle in the original answer choices is a "trigger phrase-lifestyle has been used 
over and over by people who are homophobic. So has the issue of choice" (Gabi Clayton, 1998, in e-mail reply). 
Most importantly, she provided crucial input to assure maximum fidelity. Her story seemed appropriate for use is this 
project for several reasons. It is a well-written account of the struggles of one family to deal with adolescent 
homosexuality, abuse of and violence towards homosexuals, depression, and teen suicide. More importantly. Bill's 
Story includes vivid accounts of dilemmas and decisions faced by Bill Clayton and his family. It is a true story in 
which a family faced problems head-on, made appropriate decisions, and provided more than adequate support. 
However, Bill's Story is also one that, despite all efforts to intervene, ends in tragedy. This contradiction is the 
dynamic diagonal in this composition, which is intended to inspire robust discourse. 

Adapting the Story to Narrative Simulation 

The next issue was development of the true story into a narrative simulation exercise. This required 
transforming it from a retrospective account to an unfolding and interactive tale. We adapted Cole's (1997) 
behavioral nwdel, which illustrates the relationship between narrative, cognition, conduct, and consequence. He 
stresses that narrative simulation can be an effective means to vicariously exp>erience consequences and may even 
have power to change behavior before the fact. This adaptation goes a step further (indicated by broken lines) to 
suggest that unless the cultural collective consequences are made personal, they may have no impact, behavioral ly 
speaking, on individual attitudes and beliefs underlying discrimination. [Insert figure 1 about here.] 

Following the work of Halpem (1984) and Cole (1997), the first task was to create a qualitatively different 
instructional environment that would encourage critical forethought (simulation) instead of informed hindsight (case 
study). The story was condensed and critical decision points were determined. Each decision point or dilemma 
required developing corresponding questions and answer choices. The initial questions were developed and reviewed 
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by various experts in the fields of Educational Psychology, Curriculum and Instruction, Instructional Design, and 
Ethics, as well as both parents and gays. These reviewers assisted by suggesting important editorial revisions. 

Using Qualitative Field Study for Content Development 

During this initial process of revision it became increasingly apparent that there was a need for the 
simulation designer (McCrary) to develop a qualitatively unique and deep understanding of the broader issues of bias 
and discrimination. Various techniques of qualitative field study were en^loyed to broaden the designer's personal 
understanding and to enrich the final product. The final tasks in development of the instructional components 
included design of questions and instructional response choices, as well as supplementary explanatory information 
for an instructor’s guide. Due to the fact that homophobia3 persists as an “invisible issue" (Prettyman, 1997, p. 92) in 
American conten^x)rary culture, understanding a range of possible user characteristics and designing response 
choices that would be informative and meaningful was a daunting task. Qualitative field study was viewed as an 
appropriate means to this end. A series of participant observations and interviews were con^leted, contributing data 
specifically on homophobia, as well as the larger topic of discrimination, including racism, sexism, anti-Semitism, 
and disability. Two major reoccurring themes surfaced throughout these formal and informal observations and 
interviews. Those were religious belief and unwillingness to claim personal bias. From a classical instructional 
design perspective, religious belief and denial of personal bias might be considered as important entry-level 
characteristics the target learner. 

For purposes of this design effort, religious beliefs may be no less important than preexisting attitudes and 
beliefs about worker safety or gender related behavior in the workplace. Diana Butler Bass (1998), associate 
professor of religious studies at Rhodes College, posits that in the cases of African American slavery, the subjugation 
of women, and anti-Semitism. "Christianity developed elaborate theological systems based on a few biblical texts to 
keep people subject to the cultural status quo and male authority (D5)". It follows that discrimination towards 
homosexuals may, in fact, be encouraged by such theological systems. Additionally cultural experience and 
background may contribute to miseducative experiences regarding race, gender, and minorities. Eisner references 
John Dewey in describing miseducative experience. “Miseducative experience arrests growth and/or develops 
dispositions towards a domain of human experience that limit or diminish the probability of growth in that area. 
(Eisner, 1998, p.99)." 

The reoccurring theme of the refusal of those observed and interviewed to claim personal prejudice also 
raised questions and design considerations. Most informants interviewed were quite comfortable discussing 
pervasive discrimination in our culture, yet, almost without exception, viewed it as the behavior of others. "But 
focusing on the prejudices of others leads us down just one path to a hostile and divided culture as each group, blind 
to its own flaws and vulnerabilities, comes to believe that it holds the moral high ground (Bennett, 1998).” This 
reiterates the problem in education of social versus personal relevance (Cole, 1997, see figure 1). Discriminatory 
behavior usually results in immediate and tangible consequences for others, rather than the perpetrators themselves. 
Therefore, situating the learner as a character in an unfolding tale through the use of engaging questions and frank 
responses may be the most crucial function of narrative simulation for this type of instructional design effort. "This 
just is! Jeffs Story employs this function in situating instructional exp»erience in an environment that encourages 
reflection, empathy, and compassion. 

Program Description 

"This just is!" Jeffs Story is an instructional program that tells a concise story. It includes an embedded 
question following each possible decision point in the narrative. After each question several possible answer choices 
are listed, as well as space for write-in answers. Graphics are included throughout as a means to appeal to aesthetic 
and sensory ways of understanding and remembering. Each page of narrative includes an isolated text box following 
the story segment that contains pertinent historical accounts, news media excerpts, and statistical data. A separate 
answer discussion key is attached, which includes questions and answers from the exercise booklet. Following each 
possible answer choice is a subjective commentary relative to that particular answer. The answer discussion key, as 
the name suggests, is designed to stimulate group discussion and individual reflection. Users are asked to con^are 
their answers with those in the answer discussion key and read the acconpanying comments. They are then 
encouraged to discuss individual opinions with the group. Since this instructional package is intended for widely 
diverse audiences and settings, an instructor's manual is provided. The instructor's manual includes additional 
information for facilitating participant discussion. It provides suggested focus for discussion and clearly outlined 
objectives. This manual is designed to prepare and guide facilitators through the instructional process. 



2 Homophobia has been defined as “irrational fear and hatred of those who love and sexually desire those of the 
same sex (Pharr, 1988. P. 1).” Comparisons have been drawn between honx>phobia, racism, and anti-Semitism in 
terms of discrimination that includes verbal and physical violence, even death (Pharr, 1988) 
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Figure 2. Sample Narrative Page 
'This just is!" Jeffs Story Instructional Simulation 



The Ucuinniiig of the Knd 

On April 4, 1997, Steven, his girlfriend, Julie, and Jeff were walking near the County High School on their way lo Julie’s to 
watch a video they had rented. Four guys from their high school were cruising by the school. They recognized Jeff and followed 
the three, yelling angry obscenities. “Queers, faggots...” 

Steven, Julie, and Jeff tried to ignore the insults and threats, deciding to cut through the school grounds to avoid the car 
following them. The guys in the car pariced and proceeded, on foot, to surround Jeff and his friends. They brutally assaulted 
Steven and Jeff, kicking and beating them unconscious. It was a bright spring afternoon, and all Julie could do was scream at 
them to stop. When Steven and Jeff regained consciousness, they went with Julie to find the school custodian. He called the 
police and an ambulance. Both boys were in serious condition with multiple abrasions, bruises, and broken bones. 



Question F: As Jeff’ s parent, you are now faced with what to do to help your son. How will you proceed? 

□ 41 . Get Jeff to a psychiatric hospital immediately. 

□ 42. Tell Jeff that this sexual orientation he has chosen almost got him killed and has brought nothing but trouble. 

□ 43. Call the minister at your church to get his advice. 

□ 44. Tell Jeff you love him and want to help him through this difficult time. 

□ 45. Talk to your friends and try to get their advice. 

□ 46. Try to find Jeff a nice girlfriend. 

□ 47. Make an appointment with a family therapist to get some professional help for you and your son. 

□ 48. Other 



Figure 3. Sample Didactic Text Box, Graphic, and Original Story Quotation 
"This just is!" Jeff s Story Instructional Simulation 




A Bit of World History 



“Who belonged in a concentration camp, according to the Gestapo? Above all four groups of people: political adversaries, members of 
inferior races and inferiors from the standpoint of race biology, criminals, and asocials. ...All groups of prisoners in the concentration 
camp had to wear external markings which were sewn on their clothing, namely a number and a triangle of a certain color on the left side 
of the chest as well as the right trouser leg. Red was the color of the political prisoners. ..The other colors and designations were as 
follows: Green for criminals... violet for Jehovah’s witnesses, black for asocials, pink for homosexuals, at times brown for gypsies... Jews 
wore an inverted yellow triangle underneath their red, green, black, or other markings, forming a star with six points.” (Kogon, 1946, 
cited from Duberman, 1989, p.376) 

[Jeff wrote this in his journal! His brother found it when he was going through Jeffs belongings 

after the funeral.] 



Evaluation of ”This just isP* JefTs Story 
Focus of Field Testing 

An initial field test of the narrative simulation was conducted with two user groups. These included a 
graduate class of professionals in the field of Higher Education Student Development at a large southeastern 
university and a small group of ministers certified as pastoral counselors working in a large urban area in the 
northeast. Since the simulation will have to be promoted by facilitators or those engaged in raising awareness on the 
issues addressed in the narrative simulation, these groups of users were selected for initial user tests. 

The design features prioritized for field-testing were usability, impact on user attitudes, and relevance of 
didactic information embedded throughout the exercise. We wanted, first, to determine the overall usability for ease 
in following instructions, answering questions, and cohesiveness of narrative conposition. To this end we used a 
participant evaluation that was designed to be filled-out immediately after completion of the exercise. Additionally 
participants were asked to fill-out another very brief evaluation following the review of the answer discussion key 
and subsequent group discussion. This served to provide specific information about the value of group discussion. 
Cohesiveness of the composition was determined through careful analysis of participant answer choices, with 
particular attention to user write-in answers. The next priority was to collect preliminary data on the potential of this 
exercise to impact user' attitudes regarding homosexuality. This was done through a comparison of initial participant 
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evaluations with a follow-up evaluation, which was con^leted at least two weeks after the simulation experience. We 
considered time for reflection an in^wrtant factor in attitude change, recognition of impact on beliefs, and ability to 
articulate such. Finally, we wanted to determine the relevance of the didactic information embedded throughout the 
narrative simulation. This information, appearing in text boxes, was evaluated through overall analysis of participant 
answers and comments on various participant evaluations, as well as comments during group discussion. 

The higher education development professionals used the simulation exercise during one graduate class 
period, while the ministers used it during a quarterly ministerial meeting. The narrative simulation packet was 
distributed and users were instructed to work through the simulation. Immediately following individual use of the 
simulation packet, users filled out a brief evaluation that included questions regarding usability, veracity, informative 
aspects, and intact on attitudes and beliefs. The participants then worked in small groups to discuss their responses 
and the narrative itself. Following the group work, users were asked to respond to several questions specifically 
regarding the intact of group discussion. Finally, the designer debriefed the users in an open-ended group discussion 
to elicit any additional feedback not addressed by the evaluation questionnaire. We also atten^)ted to contact users 
after several weeks in an effort to ascertain residual affects. Although the initial field tests include fourteen users, we 
were only able to obtain eight returned follow-up evaluations. The follow-up contact occuned at least two weeks 
after the simulation experience. It includes ten statements similar to those in the initial evaluation. 

Formative Evaluation Results 

Table 1 summarizes the participant evaluations and includes percentage results for eleven questions 
regarding the individual experience and four questions relative to the group discussion. These were rated from one to 
five (5 = highest agreement) on a Likert scale. As part of our analysis, we categorized these statements under six 
areas: 1) veracity, 2) general usability, 3) the extent to which the user felt the experience was informative, 4) 
cognitive or the extent to which the exercise provoked thinking, 5) impact on attitude, and 6) the value of group 
discussion in the exercise process. These findings are summarized and reported in terms of general agreement with 
specific evaluation questions as indicated by user choice of either number (4) or (5) on the scale, disagreement by 
choice of number (1) or (2), and those choosing number (3) were viewed as neutral. 

Table 1. Initial User Evaluation Results ”This just is!” Jeffs Story {n = 14) 



Simulation Evaluation Questions 


Design 

Category 


Agree 

(#4.5) 


Disagree 

(#1,2) 


Neutral 

(#3) 


1 could relate personally to this story. 


Veracity 


28% 


28% 


36% 


Jeff s Story was realistic. 


Veracity 


86% 


0% 


7% 


The questions were easy to answer. 


Usability 


57% 


43% 


0 


The directions in the exercise are easy to follow. 


Usability 


93% 


7% 


0 


I found the additional information at the bottom of some pages appropriate and informative. 


Informative 


50% 


7% 


29% 


This story gave me new information about homophobia and discrimination towards gays. 


Informative 


29% 


43% 


29% 


Prior to working this exercise I had not thought much about discrimination toward gays. 


Cognitive 


0 


100% 


0 


What 1 learned from Jeffs Story will help me respond to discrimination towards gay youth. 


Cc^itive 


50% 


14% 


36% 


The questions in this exercise were thought provoking. 


Cc^itive 


93% 


7% 


0 


1 found Jeffs Story disturbing. 


Attitude 


79% 


0 


21% 


This exercise made me feel uncomfortable. 


Attitude 


21% 


64% 


21% 


The group discussion while reviewing the answer discussion key was helpful. 


Collaboration 


86% 


0 


14% 


It was useful to hear others viewpoints regarding decision points in the exercise. 


Collaboration 


93% 


0 


7% 


The group discussion caused me to rethink some of my initial responses. 


Collaboration 


71% 


14% 


14% . 


The group discussion enhanced my experience with this exercise. 


Collaboration 


86% 


0 


14% 



Usability 

Two specific evaluation items related to general usability. Those were statements regarding following the 
exercise instructions and relative ease of answering the embedded questions throughout the exercise.. 93% of users 
indicated that the instructions were easy to follow, while 57% agreed that the questions were easy to answer. During 
group discussion it was noted that one reason users said exercise questions were difficult was because they struggled 
with the dilemmas presented. Written comments regarding general usability were as follows. "Good exercise, very 
interactive." "It’s a wonderful concept and the tool is well designed!" "Excellent investigative tool." 

Veracity 

Two statements in the participant evaluation related specifically to truth-likeness of the narrative. Although 
participants were told that this exercise was based on a true story, we wanted to know whether they personally 
perceived it as believable. Thus one stateirient concerned the extent to which users could relate personally to the 
story, while the other singly stated that "*This Just is!'' Jeffs Story was realistic. Only 28% indicated that they could 
relate personally, with 28% claiming inability to relate and 36% remaining neutral. However, 86% of participants 
indicated that the simulation was realistic. There were no written comments specific to the category of veracity. 
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Informative 

One evaluative statement relative to the informative nature of the simulation dealt with appropriateness of 
the additional information provided in text boxes throughout the narrative. A second statement in this category 
related to whether the story itself provided new information about homophobia and discrimination toward gays. We 
wanted to know if such a contemporary story was familiar to users, as well as determine whether the history, news 
media information, and statistics provided was noticed and to what extent it appropriately informed. The participants 
in these two initial field tests indicated that while the story did not provide new information on homophobia or 
discrimination (only 29% agreed), the text box information was appropriate and informative for 50% of these users. 
Two written comments relative to this category were especially helpful in our thinking about further development. 
One user wrote, ’’already had experience with homophobia”. Another noted that the historical information "seemed 
one-sided”. The debriefing discussion clarified this issue. Users noted that more geographically relevant examples 
would be pertinent. 

Cognitive 

The statements in this category pertained to users’ sense of their own thinking process with regard to the 
issues at hand. These were designed to get a sense of whether participants felt they had thought about this topic 
previously compared to the extent to which this experience would provoke cognitive processing. We also used a 
statement about whether users’ felt that the thinking stimulated through the exercise would impact personal behavior 
relative to discrimination. The issue of whether participants had previously thought about discrimination towards 
gays reflected 100% disagreement. That is, all users believed they were aware of this issue before participating in the 
field test. 50% indicated that this experience would ultimately change their behavior with regard to discrimination 
towards gays, and 93% said the narrative simulation was thought provoking. 

Affective 

This category consisted of two statements intended to understand the extent to which this exp>erience was 
disturbing and made participants feel uncomfortable. Our agenda here was to produce some way to begin to 
determine optimal balance between necessary discomfort for narrative effectiveness (cognitive dissonance) and 
residual negative feelings as a result of undue discomfort. Interestingly enough, 79% of users said, "This Just is!" 
Jeffs Story was disturbing and only 21% claimed that it made them feel uncomfortable. 

Group Discussion 

We felt that one of the most important factors for productive impact of this instructional design would be 
group discussion. In that spirit, we emphasized collaboration by including four statements regarding group process in 
this instructional experience. These statements had to do with whether group discussion was helpful, whether it was 
useful to hear others viewpoints, whether group discussion caused rethinking of initial responses, and whether group 
discussion provided overall enhancement of the experience. Agreement on these issues was, respectively, 86%, 93%, 
71%, and 86%, leaving only a small minority of participants neutral or in disagreement. 

Foilow-up Evaluation 

Table # 2 shows the results of the participant follow-up evaluation. The general themes of these statements 
reflect issues similar but not identical to the initial instrument. These themes included, broader understanding, 
stimulation of further discussion of homophobia and discrimination towards gays, utilization of information retained, 
actual change in behavior, continued thinking on the subject, change in attitude after reflection, lingering 
disturbance, increased awareness and sensitivity, retention of information gained from group discussion, and overall 
satisfaction with having had the experience. One striking difference in the follow-up results and those of the initial 
evaluation was that more respondents indicated a neutral position on many of the statements in the follow-up scale. 
63% of participants indicated a broader understanding of homophobia after the simulation experience, 50% said they 
had discussed these issues further, 88% suggested that the experience continued to be thought provoking, 63% said 
they still found the exercise disturbing, and 76% were glad they had participated. In disagreement with the 
statements 63% indicated that they had not yet been able to utilize the information gained from the exercise and 51% 
said they did not feel differently than they did immediately after the experience. 
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Table 2. Follow-up User Evaluation Results ''This just is!** Jeff s Story (n = 8) 





Design Category 


Agree 

(#4.5) 


Disagree 

m2) 


Neutral 

(#3) 


I have already been able to utilize the new information presented in this exercise. 


Behavioral 


13% 


63% 


25% 


What 1 learned from Jeffs Story has helped me respond better to discrimination towards gay 
youth. 


Behavioral 


38% 


25% 


38% 


1 have discussed my thinking on this topic with others since working the exercise. 


Cognitive 


50% 


38% 


13% 


1 find this experience has broadened my understanding of homophobia. 


Cognitive 


63% 


25% 


13% 


The questions in this exercise continue to be thought provoking for me. 


Cognitive 


88% 


13% 


0 


After reflection I felt differently about this experience than I did immediately after working 
the exercise. 


Attitude 


38% 


51% 


13% 


1 am more aware and sensitive to broad issue of discrimination since my experience with 
Jeffs Story. 


Attitude 


26% 


26% 


50% 


1 am glad 1 had the experience using this exercise. 


Attitude 


76% 


13% 


13% 


1 still find Jeffs Story disturbing. 


Attitude 


63% 


25% 


13% 


I remember more from the group discussion than the actual exercise. 


Collaboration 


25% 


38% 


38% 



Reflections on Instructional Narrative Simulation 

What are the implications for the use of nanrative simulation to address difficult topics such as bias and 
discrimination? To what extent can the use of qualitative development strategies and ethnographic field study 
methodologies inform the design of such instructional materials? The results of preliminary field-tests are 
encouraging in several areas. First, the transformation of a true story case into an instructional narrative simulation 
was possible and indeed achieved some of the desired design intentions (character/circumstance identification and 
user engagement). Secondly, embedding didactic information along with narrative informed and served to further 
frame the story as contemporary. It appears that the embedded didactic information accomplished these objectives 
without eroding the flow of the story or the involvement for the majority of users in this group. Third, the 
combination of the narrative simulation with group discussion appears to have potential to greatly increase the 
impact of the experience. This is particularly important because it is precisely the lack of public discussion and 
disclosure of personal belief that appear to be key issues in addressing the so-called “invisible” aspects of 
homophobia and other forms of discrimination. Fourth, This experience of the simulation appears to have affects 
beyond the initial use of the exercise, as indicated by responses to the follow-up evaluation. Finally, the results of 
this initial user test suggest several improvements in the design. Those include the use of more geographically 
diverse didactic information to promote user identification and revisions to the instructor’s manual regarding reaction 
to user responses. 

These results engender a commitment to look carefully at responses to questions on additional tests with 
other user groups (parents, adolescents, etc.), in order to make informed judgments regarding retaining questions that 
are never selected or may otherwise detract from the power of the narrative. Lastly, through the design process for 
the narrative simulation, we have found the use of qualitative methods in the development process has much promise. 
Such field study immerses the designer in the context of instmetion and also provides rich, contemporary insights 
that can be folded back into the design. 
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